{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 感知机原理小结\n",
    "\n",
    "## 一、感知机算法简介\n",
    "\n",
    "1. 一条直线，将数据分开，如男女生分类。\n",
    "2. 使用感知机的前提，数据是线性可分的。如果不可分，支持向量机通过核技巧让数据高维可分，神经网络通过激活函数和增加隐藏层让数据可分。\n",
    "3. 超平面不唯一，感知机可以有多个解。\n",
    "\n",
    "## 二、算法描述\n",
    "\n",
    "### 2.1 背景\n",
    "\n",
    "m个样本，每个样本n维特征对应一个二元类别输出：\n",
    "$$\n",
    "(x_1^{(0)}, x_2^{(0)}, ...x_n^{(0)}, y_0), (x_1^{(1)}, x_2^{(1)}, ...x_n^{(1)},y_1), ... (x_1^{(m)}, x_2^{(m)}, ...x_n^{(m)}, y_m)\n",
    "$$\n",
    "找到一个超平面：\n",
    "$$\n",
    "\\theta_0 + \\theta_{1}x_1 + ... + \\theta_{n}x_{n} = 0\n",
    "$$\n",
    "使得一类样本满足：$\\theta_0 + \\theta_{1}x_1 + ... + \\theta_{n}x_{n} > 0$，另一类满足：$\\theta_0 + \\theta_{1}x_1 + ... + \\theta_{n}x_{n} < 0$从而得到线性可分。超平面不唯一。\n",
    "\n",
    "简化写法，增加一个$x_0 = 1$,$\\sum\\limits_{i=0}^{n}\\theta_{i}x_{i} = 0$，用向量表示就是：  $\\theta \\bullet x = 0$     ,$\\bullet$  表示内积，代表一条直线或者超平面\n",
    "\n",
    "其中$θ$为(n+1)x1的向量，$x$为(n+1)x1的向量\n",
    "\n",
    "### 2.2感知机模型\n",
    "\n",
    "模型定义为:\n",
    "$$\n",
    "y = sign(\\theta \\bullet x)\\\\\n",
    "sign(x)= \\begin{cases} -1& {x<0}\\\\ 1& {x\\geq 0} \\end{cases}\n",
    "$$\n",
    "\n",
    "### 2.3 损失函数\n",
    "\n",
    "分类正确的样本：\n",
    "$$\n",
    "y\\theta \\bullet x > 0\n",
    "$$\n",
    "分类错误的样本：\n",
    "$$\n",
    "y\\theta \\bullet x < 0\n",
    "$$\n",
    "对于每一个分错的样本来说，到超平面的距离，$||\\theta||_2$是$L2$范式，对于所有分类错误的样本，M为分类错误的集合：\n",
    "$$\n",
    "\\frac{ - y^{(i)}\\theta \\bullet x^{(i)}}{||\\theta||_2}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\frac{ - \\sum\\limits_{x_i \\in M}y^{(i)}\\theta \\bullet x^{(i)}}{||\\theta||_2}\n",
    "$$\n",
    "\n",
    "分子分母都含有$\\theta$，分子分母有固定的倍数关系。\n",
    "\n",
    "那么我们可以固定分子或者分母为1，然后求另一个即分子自己或者分母的倒数的最小化作为损失函数，这样可以简化我们的损失函数。\n",
    "\n",
    "在感知机模型中，我们采用的是保留分子，即最终感知机模型的损失函数简化为：\n",
    "\n",
    "\n",
    "$$\n",
    "J(\\theta) = - \\sum\\limits_{x_i \\in M}y^{(i)}\\theta \\bullet x^{(i)}\n",
    "$$\n",
    "\n",
    "### 2.4 梯度下降优化\n",
    "\n",
    "只有误分类集合M才能参与损失函数的优化，因此不能采用批梯度下降。这里采用SGD:\n",
    "$$\n",
    "\\frac{\\partial}{\\partial \\theta}J(\\theta) = - \\sum\\limits_{x_i \\in M}y^{(i)}x^{(i)}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\theta = \\theta  + \\alpha\\sum\\limits_{x_i \\in M}y^{(i)}x^{(i)}\n",
    "$$\n",
    "\n",
    "SGD每次只对一个误分类样本进行梯度下降：\n",
    "$$\n",
    "\\theta = \\theta  + \\alpha y^{(i)}x^{(i)}\n",
    "$$\n",
    "其中$α$为步长，$y_{(i)}$为样本输出1或者-1，$\\theta 、x_{(i)}$为(n+1)x1的向量。 \n",
    "\n",
    "最终检查训练集是否还有误分类点，没有，算法结束。有的话，继续进行梯度下降优化\n",
    "\n",
    "## 三、代码实现验证\n",
    "\n",
    "不同角度理解感知机：\n",
    "\n",
    "1. [零基础入门深度学习(1) - 感知器](https://www.zybuluo.com/hanbingtao/note/433855)\n",
    "2. [超简单感知机及代码实现](https://www.jianshu.com/p/428b29d4a511) \n",
    "3. [感知机原理小结 刘建平](https://www.cnblogs.com/pinard/p/6042320.html)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wrong Point {'x': array([1, 1]), 'y': -1}\n",
      "update w to [[-1 -1]] \n",
      "update b to -1 \n",
      "Wrong Point {'x': array([3, 3]), 'y': 1}\n",
      "update w to [[2 2]] \n",
      "update b to 0 \n",
      "Wrong Point {'x': array([1, 1]), 'y': -1}\n",
      "update w to [[1 1]] \n",
      "update b to -1 \n",
      "Wrong Point {'x': array([1, 1]), 'y': -1}\n",
      "update w to [[0 0]] \n",
      "update b to -2 \n",
      "Wrong Point {'x': array([3, 3]), 'y': 1}\n",
      "update w to [[3 3]] \n",
      "update b to -1 \n",
      "Wrong Point {'x': array([1, 1]), 'y': -1}\n",
      "update w to [[2 2]] \n",
      "update b to -2 \n",
      "Wrong Point {'x': array([1, 1]), 'y': -1}\n",
      "update w to [[1 1]] \n",
      "update b to -3 \n",
      "Wrong Point None\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "X = np.array([[1, 1], [3, 3], [4, 3]])\n",
    "y = [-1, 1, 1]\n",
    "\n",
    "class Perceptron(object):\n",
    "    def __init__(self, learning_rate=1):\n",
    "        self.w = np.array([0, 0]).reshape(-1, 1)\n",
    "        self.b = 0\n",
    "\n",
    "    def sign(self, x):\n",
    "        return -1 if x < 0 else +1\n",
    "\n",
    "    def calculate(self, X):\n",
    "        yH = np.matmul(X, self.w) + self.b\n",
    "        return np.apply_along_axis(self.sign, 1, yH)\n",
    "\n",
    "    def get_wrong(self, X, yH, Y):\n",
    "        for x, yh, y in zip(X, yH, Y):\n",
    "            if yh != y:\n",
    "                return {'x': x, 'y': y}\n",
    "        return None\n",
    "\n",
    "    def fit(self, X, y):\n",
    "        while True:\n",
    "            yH = self.calculate(X)\n",
    "            wrong = self.get_wrong(X, yH, y)\n",
    "            print(f\"Wrong Point {wrong}\")\n",
    "            if not wrong:\n",
    "                break\n",
    "            self.w = self.w + (wrong['x'] * wrong['y']).reshape((-1, 1))\n",
    "            self.b = self.b + wrong['y']\n",
    "            print(f\"update w to {self.w.reshape(1,-1)} \")\n",
    "            print(f\"update b to {self.b} \")\n",
    "            \n",
    "\n",
    "per = Perceptron()\n",
    "per.fit(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x140b0ee4248>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAUGElEQVR4nO3df4xdZ33n8fcnsVvqJCIUTyHrHzHSplIB5deOTJArSNpucFggqhppHWVDYIkssVRLdlFXJZESNYg/VkjRLmxby0uiQNcJVCShXpRAXEE3pSghY6/JLwO1QkIsR/IQ0/yoEZXT7/5xj5fL5M7MGfva43n8fklXc+7zPOfO98mTfO7JuefOSVUhSWrXaYtdgCTp+DLoJalxBr0kNc6gl6TGGfSS1Lhli13AKCtXrqx169YtdhmStGTs3LnzJ1U1MarvpAz6devWMTU1tdhlSNKSkeTZ2fo8dSNJjTPoJalxBr0kNc6gl6TGGfSS1Lh5gz7J65J8N8n3kjyZ5E9GjPnVJF9OsjfJI0nWDfV9smv/QZL3jLd8SQu1bRusWwennTb4uW3bYlek470mfS6v/DnwO1X1SpLlwLeTPFBVDw+N+Qjw06r6l0k2Af8V+LdJ3gpsAt4G/Avgr5P8ZlW9Ot5pSOpj2zbYvBkOHRo8f/bZwXOAa65ZvLpOZSdiTeY9oq+BV7qny7vHzL9tfCXwhW77K8DvJknX/qWq+nlV/QjYC6wfS+WSFuymm34RKEccOjRo1+I4EWvS6xx9ktOT7AYOADuq6pEZQ1YBzwFU1WHgReCNw+2dfV3bqN+xOclUkqnp6emFzUJSLz/+8cLadfydiDXpFfRV9WpVXQisBtYnefuMIRm12xzto37H1qqarKrJiYmR3+KVdIzWrl1Yu46/E7EmC7rqpqr+AfgbYOOMrn3AGoAky4DXAweH2zurgf1HWaukY/TpT8OKFb/ctmLFoF2L40SsSZ+rbiaSnN1t/xrwe8D3ZwzbDlzXbV8FfLMG9yjcDmzqrsp5C3Ae8N1xFS9pYa65BrZuhXPPhWTwc+tWP4hdTCdiTTLfPWOTnM/gg9bTGbwx/GVV3ZrkVmCqqrYneR3wF8BFDI7kN1XV093+NwH/HjgM3FBVD8xX1OTkZPlHzSSpvyQ7q2pyZN/JeHNwg16SFmauoPebsZLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxi2bb0CSNcAXgTcD/wxsrar/PmPMHwFHbny1DPgtYKKqDiZ5BngZeBU4PNsfxpckHR/zBj2DWwB+oqp2JTkL2JlkR1U9dWRAVX0G+AxAkvcD/6mqDg69xmVV9ZNxFi5J6mfeUzdV9XxV7eq2Xwb2AKvm2OVq4O7xlCdJOlYLOkefZB2DG4A/Mkv/CmAjcM9QcwEPJtmZZPMcr705yVSSqenp6YWUJUmaQ++gT3ImgwC/oapemmXY+4G/m3HaZkNVXQxcAXwsybtG7VhVW6tqsqomJyYm+pYlSZpHr6BPspxByG+rqnvnGLqJGadtqmp/9/MAcB+w/uhKlSQdjXmDPkmA24E9VXXbHONeD7wb+KuhtjO6D3BJcgZwOfDEsRYtSeqvz1U3G4BrgceT7O7abgTWAlTVlq7t94EHq+ofh/Z9E3Df4L2CZcBdVfX1cRQuSepn3qCvqm8D6THuTuDOGW1PAxccZW2SpDHwm7GS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1rs8dptYk+VaSPUmeTPLxEWMuTfJikt3d4+ahvo1JfpBkb5I/HvcEJElz63OHqcPAJ6pqV3dbwJ1JdlTVUzPG/W1VvW+4IcnpwJ8C/xrYBzyaZPuIfSVJx8m8R/RV9XxV7eq2Xwb2AKt6vv56YG9VPV1V/wR8CbjyaIuVJC3cgs7RJ1kHXAQ8MqL7nUm+l+SBJG/r2lYBzw2N2ccsbxJJNieZSjI1PT29kLIkSXPoHfRJzgTuAW6oqpdmdO8Czq2qC4DPAV89stuIl6pRr19VW6tqsqomJyYm+pYlSZpHr6BPspxByG+rqntn9lfVS1X1Srd9P7A8yUoGR/BrhoauBvYfc9WSpN76XHUT4HZgT1XdNsuYN3fjSLK+e90XgEeB85K8JcmvAJuA7eMqXpI0vz5X3WwArgUeT7K7a7sRWAtQVVuAq4CPJjkM/AzYVFUFHE7yh8A3gNOBO6rqyTHPQZI0hwzy+OQyOTlZU1NTi12GJC0ZSXZW1eSoPr8ZK0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqXJ9bCa5J8q0ke5I8meTjI8Zck+Sx7vGdJBcM9T2T5PEku5N4NxFJOsH63ErwMPCJqtqV5CxgZ5IdVfXU0JgfAe+uqp8muQLYCrxjqP+yqvrJ+MqWJPU1b9BX1fPA8932y0n2AKuAp4bGfGdol4eB1WOuU5J0lBZ0jj7JOuAi4JE5hn0EeGDoeQEPJtmZZPMcr705yVSSqenp6YWUJUmaQ59TNwAkORO4B7ihql6aZcxlDIL+t4eaN1TV/iS/AexI8v2qemjmvlW1lcEpHyYnJ0++O5ZL0hLV64g+yXIGIb+tqu6dZcz5wOeBK6vqhSPtVbW/+3kAuA9Yf6xFS5L663PVTYDbgT1VddssY9YC9wLXVtUPh9rP6D7AJckZwOXAE+MoXJLUT59TNxuAa4HHk+zu2m4E1gJU1RbgZuCNwJ8N3hc4XFWTwJuA+7q2ZcBdVfX1sc5AkjSnPlfdfBvIPGOuB64f0f40cMFr95AknSh+M1aSGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TG9bnD1Jok30qyJ8mTST4+YkySfDbJ3iSPJbl4qO+6JH/fPa4b9wQkSXPrc4epw8AnqmpXd1vAnUl2VNVTQ2OuAM7rHu8A/hx4R5JfB24BJoHq9t1eVT8d6ywkSbOa94i+qp6vql3d9svAHmDVjGFXAl+sgYeBs5OcA7wH2FFVB7tw3wFsHOsMJElzWtA5+iTrgIuAR2Z0rQKeG3q+r2ubrX3Ua29OMpVkanp6eiFlSZLm0Dvok5wJ3APcUFUvzewesUvN0f7axqqtVTVZVZMTExN9y5IkzaNX0CdZziDkt1XVvSOG7APWDD1fDeyfo12SdIL0ueomwO3Anqq6bZZh24EPdlffXAK8WFXPA98ALk/yhiRvAC7v2iRJJ0ifq242ANcCjyfZ3bXdCKwFqKotwP3Ae4G9wCHgw13fwSSfAh7t9ru1qg6Or3xJ0nzmDfqq+jajz7UPjyngY7P03QHccVTVSZKOmd+MlaTGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1bt4bjyS5A3gfcKCq3j6i/4+Aa4Ze77eAie7uUs8ALwOvAoeranJchUuS+ulzRH8nsHG2zqr6TFVdWFUXAp8E/s+M2wVe1vUb8pK0COYN+qp6COh7n9ergbuPqSJJ0liN7Rx9khUMjvzvGWou4MEkO5Nsnmf/zUmmkkxNT0+PqyxJOuWN88PY9wN/N+O0zYaquhi4AvhYknfNtnNVba2qyaqanJiYGGNZknRqG2fQb2LGaZuq2t/9PADcB6wf4++TJPUwlqBP8nrg3cBfDbWdkeSsI9vA5cAT4/h9kqT++lxeeTdwKbAyyT7gFmA5QFVt6Yb9PvBgVf3j0K5vAu5LcuT33FVVXx9f6ZKkPuYN+qq6useYOxlchjnc9jRwwdEWJkkaD78ZK0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuPmDfokdyQ5kGTk3aGSXJrkxSS7u8fNQ30bk/wgyd4kfzzOwiVJ/fQ5or8T2DjPmL+tqgu7x60ASU4H/pTBjcHfClyd5K3HUqwkaeHmDfqqegg4eBSvvR7YW1VPV9U/AV8CrjyK15EkHYNxnaN/Z5LvJXkgydu6tlXAc0Nj9nVtIyXZnGQqydT09PSYypIkjSPodwHnVtUFwOeAr3btGTG2ZnuRqtpaVZNVNTkxMTGGsiRJMIagr6qXquqVbvt+YHmSlQyO4NcMDV0N7D/W3ydJWphjDvokb06Sbnt995ovAI8C5yV5S5JfATYB24/190mSFmbZfAOS3A1cCqxMsg+4BVgOUFVbgKuAjyY5DPwM2FRVBRxO8ofAN4DTgTuq6snjMgtJ0qwyyOSTy+TkZE1NTS12GZK0ZCTZWVWTo/r8ZqwkNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXHzBn2SO5IcSPLELP3XJHmse3wnyQVDfc8keTzJ7iTeSUSSFkGfI/o7gY1z9P8IeHdVnQ98Ctg6o/+yqrpwtjufSJKOr3nvGVtVDyVZN0f/d4aePgysPvayJEnjMu5z9B8BHhh6XsCDSXYm2TzXjkk2J5lKMjU9PT3msiTp1DXvEX1fSS5jEPS/PdS8oar2J/kNYEeS71fVQ6P2r6qtdKd9JicnT747lkvSEjWWI/ok5wOfB66sqheOtFfV/u7nAeA+YP04fp8kqb9jDvoka4F7gWur6odD7WckOevINnA5MPLKHUnS8TPvqZskdwOXAiuT7ANuAZYDVNUW4GbgjcCfJQE43F1h8ybgvq5tGXBXVX39OMxBkjSHPlfdXD1P//XA9SPanwYueO0ekqQTyW/GSlLjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1LheQZ/kjiQHkoy8Q1QGPptkb5LHklw81Hddkr/vHteNq3BJUj99j+jvBDbO0X8FcF732Az8OUCSX2dwR6p3MLhf7C1J3nC0xc5r2zZYtw5OO23wc9u24/arJGmp6BX0VfUQcHCOIVcCX6yBh4Gzk5wDvAfYUVUHq+qnwA7mfsM4etu2webN8OyzUDX4uXmzYS/plDeuc/SrgOeGnu/r2mZrH7+bboJDh3657dChQbskncLGFfQZ0VZztL/2BZLNSaaSTE1PTy+8gh//eGHtknSKGFfQ7wPWDD1fDeyfo/01qmprVU1W1eTExMTCK1i7dmHtknSKGFfQbwc+2F19cwnwYlU9D3wDuDzJG7oPYS/v2sbv05+GFSt+uW3FikG7JJ3ClvUZlORu4FJgZZJ9DK6kWQ5QVVuA+4H3AnuBQ8CHu76DST4FPNq91K1VNdeHukfvmmsGP2+6aXC6Zu3aQcgfaZekU1SqRp4yX1STk5M1NTW12GVI0pKRZGdVTY7q85uxktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEn5eWVSaaBZ4/hJVYCPxlTOYuplXlAO3NpZR7gXE5GxzKPc6tq5J8VOCmD/lglmZrtetKlpJV5QDtzaWUe4FxORsdrHp66kaTGGfSS1LhWg37rYhcwJq3MA9qZSyvzAOdyMjou82jyHL0k6RdaPaKXJHUMeklq3JIN+iR3JDmQ5IlZ+pPks0n2JnksycUnusY+eszj0iQvJtndPW4+0TX2lWRNkm8l2ZPkySQfHzHmpF+XnvNYEuuS5HVJvpvke91c/mTEmF9N8uVuTR5Jsu7EVzq3nvP4UJLpoTW5fjFq7SvJ6Un+b5Kvjegb75pU1ZJ8AO8CLgaemKX/vcADDO5bewnwyGLXfJTzuBT42mLX2XMu5wAXd9tnAT8E3rrU1qXnPJbEunT/nM/stpcDjwCXzBjzH4At3fYm4MuLXfdRzuNDwP9Y7FoXMKf/DNw16t+jca/Jkj2ir6qHgLnuVnUl8MUaeBg4O8k5J6a6/nrMY8moquerale3/TKwB1g1Y9hJvy4957EkdP+cX+meLu8eM6/AuBL4Qrf9FeB3k+QEldhLz3ksGUlWA/8G+PwsQ8a6Jks26HtYBTw39HwfS/Q/VuCd3f+yPpDkbYtdTB/d/2pexODIa9iSWpc55gFLZF26UwS7gQPAjqqadU2q6jDwIvDGE1vl/HrMA+APulOCX0my5gSXuBD/DfgvwD/P0j/WNWk56Ee9+y3FI4BdDP6GxQXA54CvLnI980pyJnAPcENVvTSze8QuJ+W6zDOPJbMuVfVqVV0IrAbWJ3n7jCFLYk16zON/A+uq6nzgr/nFEfFJJcn7gANVtXOuYSPajnpNWg76fcDwO/pqYP8i1XLUquqlI//LWlX3A8uTrFzksmaVZDmDcNxWVfeOGLIk1mW+eSy1dQGoqn8A/gbYOKPr/69JkmXA6zmJTyfONo+qeqGqft49/Z/AvzrBpfW1AfhAkmeALwG/k+R/zRgz1jVpOei3Ax/srvK4BHixqp5f7KIWKsmbj5ybS7KewZq9sLhVjdbVeTuwp6pum2XYSb8ufeaxVNYlyUSSs7vtXwN+D/j+jGHbgeu67auAb1b3KeDJos88ZnzW8wEGn62cdKrqk1W1uqrWMfig9ZtV9e9mDBvrmiw72h0XW5K7GVz5sDLJPuAWBh/QUFVbgPsZXOGxFzgEfHhxKp1bj3lcBXw0yWHgZ8Cmk+0/wiEbgGuBx7tzqQA3AmthSa1Ln3kslXU5B/hCktMZvBn9ZVV9LcmtwFRVbWfwpvYXSfYyOGrctHjlzqrPPP5jkg8AhxnM40OLVu1ROJ5r4p9AkKTGtXzqRpKEQS9JzTPoJalxBr0kNc6gl6TGGfSS1DiDXpIa9/8ARL97BxxZA9kAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(X[:1,0],X[:1,1],color = 'red')\n",
    "plt.scatter(X[1:,0],X[1:,1],color = 'blue')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
